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ABSTRACT 


This  report  considers  the  estimation  errors  involved  in  both  discrete  and 

continuous  estimates  of  certain  parameters  of  a  Gaussian  random  process.  For 

discrete  estimates,  the  confidence  interval  concept  is  used  to  obtain  probabilistic 

bounds  on  the  estimation  errors.  Roughly  analogous  results  are  also  obtained  for 

continuous  estimates.  The  bounds  obtained  are  useful  for  a)  determining  the  accuracy 

of  an  estimate  given  the  value  of  the  estimate  and  the  number  of  samples  used  (or  for 

the  continuous  case  the  effective  TW  )  and,  b)  for  determining  roughly  the  number  of 

s 

samples  required  (or  the  effective  TW  )  to  provide  an  estimate  of  a  specified  accuracy. 

s 

The  bounds  are  presented  graphically  and  examples  of  their  use  are  given. 

A  second  result  is  the  derivation  of  an  approximate,  but  convenient  and  reasonably 
accurate  method  for  evaluating  the  non-central  t-distribution  by  means  of  tables  of  the 
normal  distribution.  This  allows  certain  calculations  to  be  made  that  are  not  now 
possible  with  existing  tables. 


AST  IA 
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I.  INTRODUCTION 


In  many  applications  it  is  important  to  estimate  certain  parameters  of  a  random 
process  such  as  its  mean  or  variance.  For  practical  reasons  these  estimations  must  be 
based  on  finite  data  records  and  thus  are  subject  to  random  errors.  This  report  is 
concerned  with  the  magnitude  of  these  errors  when  the  process  may  be  considered  to  be 
stationary  and  Gaussian. 

Two  methods  of  parameter  estimation  will  be  considered  in  this  analysis.  First, 
estimates  based  on  n  independent  samples  of  the  process  will  be  investigated.  Probabil¬ 
istic  bounds  on  the  estimation  errors  will  be  obtained  through  the  use  of  the  confidence 
interval  concept.  Secondly,  estimates  based  on  a  continuous  time  average  will  be 
considered  and  corresponding  probabilistic  error  bounds  obtained. 

II.  DISCRETE  ESTIMATION  ERRORS 

Denote  the  n  samples  on  which  the  estimate  is  to  be  based  by  Xj,  x2,  --  xn,  the 
true  value  of  the  parameter  to  be  estimated  by  6,  and  the  estimate  of  6  by 

a  a 

0  *  ©  (xj,  X2,  --  Xn).  Assume  that  the  only  unknown  parameter  in  the  known 
distribution  of  the  independent,  identically  distributed  Xj  is  ©  and  denote  the 
corresponding  density  function  by  p^x;©).  Then  the  density  function  for  §,p2(§;©), 
may  be  determined  as  a  function  of  the  unknown  parameter  6.  The  question  then  arises 

a 

as  to  whether  or  not  p2(©;0)  can  be  used  to  make  a  probabilistic  statement  concerning 
the  value  of  ©  given  a  value  of  ©.  The  answer  to  this  question  involves  two  fundamentally 
different  concepts  depending  upon  the  nature  of  ©  in  the  process  being  sampled. 

These  concepts  are  discussed  in  the  following  paragraphs. 

Consider  first  a  situation  in  which  a  large  number  of  random  processes  are 
available  each  having  a  different  value  of  ©.  An  estimate  of  6  is  to  be  obtained  by 
first  randomly  selecting  a  particular  process  and  then  taking  n  independent  samples 
of  a  sample  function  in  this  process.  After  calculating  ©  from  these  samples  it  is 
desired  to  make  a  statement  of  the  type  Pr{a<©<b}  =  1-e,  where  0  <  c  <  1). 

For  this  situation  such  a  statement  is  readily  obtained  by  considering  ©  to  have  a 
known  density  function  p(©),  noting  that  p2(©;©)  is  the  conditional  density  of  §  given 
©.  and  applying  Bayes'  theorem  (1]  to  obtain 


where 


r  rh 

P  {a<0<b  }  ■  /  p  (0/0)  de 

a 


?<§;©)  P<©) 

P<e/e)  =  ■— - 

fp  (0;e)  P(0)d0 


Note  that  this  is  just  the  conditional  probability  of  a  <  0  <  b  given  a  value  of  0. 

In  general  p2  (0;0)  will  involve  n,  the  number  of  samples,  in  such  a  way  that 
with  a  <  ©  and  b  >  0 


lim  P  {a<0<b}  =  1 
n—  *> 

while  simultaneously  b  -  a  is  approaching  zero.  Thus,  this  approach  allows  a 
probabilistic  statement  to  be  made  concerning  the  error  in  estimating  0  by  6  when 
n  is  specified  and  conversely  allows  n  to  be  determined  to  give  a  specified  error. 

Usually,  however,  estimates  are  to  be  made  using  samples  taken  from  a  sample 
function  of  a  single  process  in  which  the  parameter  0  is  an  unknown  constant.  In  this 
case  p(0)  =  6  (0  -  c )  (where  c  is  the  unknown  constant  value  of  ©  and  6  (x)  is  the  unit 
dirac  delta  function)  and  the  above  method  leads  to  the  meaningless  result 


P.  {a  <  ©  <b}  * 


{ 


1  if  a  <6  <  b 
0  if  ©  <  a  or  9  >  b 


Consideration  of  this  problem  has  led  to  the  development  of  the  theory  of 
confidence  intervals  by  J.  Neyman^  ^ .  The  following  paragraphs  discuss  this  theory 
as  it  applies  to  the  analysis  of  this  report. 

In  the  density  function  p2(@;  ©)  defined  above  substitute  an  arbitrary  number, 
say  ©' ,  for  the  unknown  parameter  0.  p2(S;0')  is  then  completely  defined  and 

probability  statements  of  the  type 


2. 
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(1) 


Pr(y<6</3}  ■  /  P2  (§;©’)  d@« 

y 

with 

/  p2 ($;©’)  d$  - 

—  00 

00 

/  p,(6;9’)d0  -  €  -  €. 

fi  2 

may  be  inferred  for  any  value  of  6'.  Now ,  as  9'  and/or  e  is  varied  in  these 
expressions  y  and  0  will  vary.  Thus  y  ■  y(9' ,  c)  and  0  ■  p  (©’ ,  c).  Assuming  y  and 
0  to  be  monotonic  functions  of  6'  these  may  be  plotted  in  the  §,  O'  plane  as  shown 
below.  1 


With  9'  *  9,  the  true  value  of  the  parameter  to  be  estimated,  and  0*0',  a 
value  of  0  determined  from  §(x^  --xq)  and  n  specific  sample  values,  consider  the 
relations 


3. 


(a)  y1<  §'  <Pl',  (b)  ©j  <  ©  <  ©2 

where  y.,y~,  ©. ,  0.  are  defined  by  the  values  of  ©and©'  as  indicated  above.  Now, 

1  Z  1  z 

for  any  particular  ©,  the  above  plot  shows  that  a  value  of  ©'  satisfying  relation  (a) 
also  satisfies  relation  (b).  Conversely,  if  ©'  satisfies  (b)  then  (a)  is  satisfied.  From 
this  the  important  conclusion  is  drawn  that  the  statement  y  ^  <  ©'  <  f}  ^  is  completely 
equivalent  to  ©^  <  0  <  ©2  and  thus  that 

Pr  {  ?!  <  *  </Jx  }  •  Pr{  <  e  <  e2  }  -  i  -  *  (2) 

It  is  in  the  interpretation  of  this  statement  that  the  difference  between  this  approach 
and  that  using  Bayes'  theorem  becomes  apparent. 

From  the  above  graph  it  is  observed  that 

*  P  (©i.O  -  r<e2,  €) 

Thus  ©j  and  ©2  are  random  variables  whose  value  depends  upon  the  values  of  the  n 
samples  used  to  obtain  the  estimate  £'  .  In  words  the  above  probability  statement  thus 
becomes  "the  probability  that  the  random  interval  [  ©^,  ©2  J  will  cover  true 
parameter  value  ©  is  1  -  e".  This  is  to  be  contrasted  to  the  Bayes'  theorem  statement 
which  reads  "  the  probability  that  the  random  variable  6  lies  in  the  fixed  interval 
[a.b]  is  1  -  €." 

In  general ,  ©^  and  ©2  will  be  functions  of  n  in  such  a  manner  that 

lim  P  {©,<©<  e„  }  -  1 

r  1  1  2  J 

n  —  *° 

Thus,  for  a  given  e  and  n  it  will  be  possible  to  make  a  probabilistic  statement  concerning 
the  estimation  error  by  the  a  priori  statement:  "before  the  n  samples  are  taken  the 
probability  that  the  interval  constructed  using  these  samples,  (i.e.  [  ©  ,  ©  )  where 

X  it 

&  *  /3(©j,  e)  «  y(©2,  c)) covers  the  true  parameter  value  ©  is  1  -  c." 


4. 


Since  the  values  and  © 2  are  random  variables  the  interval  will  also  be  a 

random  variable  from  one  set  of  samples  to  the  next.  However,  when  n  is  sufficiently 
large  to  give  useful  estimates ,  the  length  of  the  interval  varies  by  a  negligible  amount 
from  sample  to  sample.  Because  of  this  it  is  possible  to  obtain  an  accurate  indication 
of  the  magnitude  of  the  estimation  error  that  will  be  exceeded  only  100  c  %  of  the  time. 

The  interval  [©^  ©2]  is  called  the  confidence  Interval  for  the  parameter  ©, 

©^  and  ©^  are  called  the  confidence  limits,  and  1  -  e  the  confidence  level.  The  following 
paragraph  describes  a  general  method  for  determining  the  confidence  limits  and  thus 
the  confidence  interval ,  for  a  given  confidence  level. 

Prom  above,  y  (©',  c)  is  defined  by 

fy 

f  p2 (©;©’)  d©  -  €x 
-00 

where  c,  and  p2  (§;©')  are  specified.  Also,  from  the  graph  ©2  is  the  value  of  O'  In 
y  (O',  c)  which  makes  y  ■  S’.  Thus,  the  upper  confidence  limit  ©2  is  the  value  of 
©'  in  p2  (6 ;  ©’ )  such  that 

§' 

L  p2<6;e'>d®  •  «i  <*> 

In  an  analogous  manner  the  lower  confidence  limit  ©^  is  the  value  of  6  in  p2(§;  O') 
such  that 


£  p2(§; ©’) d©  -  c-c1  (4) 

where  1  -  c  is  the  desired  confidence  level,  typically  0.95  or  0. 99.  Clearly,  the 
length  of  the  confidence  interval,  ©2  -  ©, ,  will  depend  upon  and  will  have  at  least 
(me  minimum  for  0<«A<  c.  In  general,  however,  it  is  quite  difficult  to  determine  this 
extremum  and  the  usual  practice  is  to  take  «  ^  ■  c/2  which  is  the  optimum  vslue  when 
P2  (§ ;  O')  is  symmetric  in  $  and  O'.  Following  this  procedure  ©^  and  02  are  given 
by  the  implicit  relations 


5. 


(5) 


p2(e;e2)de 


€/2 


or  in  more  convenient  notation 


P  {  ©  <  0'  |  ©.  }  -  e/2 
r  2 


(5a) 


and 


or 


i*  P2  (6;©i)d6  -  e/2 
0* 


Pr  {  §  >  §'  |  ©x  }  -  e/2 


(6) 


(6a) 


Note  that  Eqs.  (5a)  and  (6a)  are  not  conditional  probabilities  but  are  simply  shorthand 
statements  for  the  previous  equations. 

1.  Confidence  Intervals  for  Probability  of  Error 

In  this  section  confidence  intervals  for  the  estimation  of  probability  of  error 
will  be  derived  by  considering  the  n  samples  to  be  drawn  independently  from  a 
discrete  random  variable  having  the  binomial  distribution.  In  all  of  the  remaining 
sections  the  samples  will  be  assumed  to  be  normally  distributed. 

Consider  the  occurence  of  errors  in  a  digital  receiver  to  be  independent  from 
baud  to  baud  and  define  a  random  variable  x  by 


x  *  1  if  an  error  occurred  in  a  given  baud, 
x  ■  0  if  no  error  occurred  in  a  given  baud. 

The  following  relations  are  then  readily  determined. 

P(x)  *  (Pd*(  1  -  Pe)l  x  x  -  0,  1  (7) 


6. 


and 

p  <*»  ■  CO  <Pe>nPe  -  *>n<1'h)  *  :  <8> 

where 

/  n^  \  _  ^  n  • 

\  n  Re'  (n  fce)'.  (n-n  Pe)'. 

Pe  =  true  probability  of  error 

A 

Pe  =  estimate  of  Pe 


i  n 

T  l 


X,  - 

n  i 

i*l 


number  of  errors  in  n  bauds 


The  xjs  are  n  independent  samples  of  the  random  variable  x  of  Eq.  (7). 

Analogous  to  Bq.  (5),  [2]  the  upper  confidence  limit  Pe^  is  the  value  of  Pe  such 

that 

A 

2  ( "fe)  <w“ *  u  -  wn(1‘fe>  i  |  m 

Pe=0  nre  1 

Similarly  the  lower  confidence  limit  Pe^  is  the  value  of  Pe  such  that 
1 

^-P*e  (nnh)  ?e)n  (1  -  Pe)n<1  *  h)  »  §  (10) 

Reference  [4]  gives  confidence  intervals  determined  from  Bqs.  (9)  and  (10)  that  are 
useful  for  Pe  >0.05  and  for  n*  10,  15,  20,  30,  50,  100,  250,  and  1000.  Usually,  how- 

A 

ever,  the  values  of  Pe  of  interest  are  less  than  0.05.  Since  tables  of  the  binomial 
distribution  are  not  available  for  Pe<  0. 01  and  n  >  1000  the  normal  approximation  to  the 
binomial  distribution  has  been  used.  With  0  <  a  <  b  <  1  this  approximation  is  [5] 


|„U)  <*>"%- *>"(1 ^ /u 


1  e‘1/2t  dt  (11) 


where 


a  -  Pe 

U1  “  ( Pe(l  -  Pe)/h] 


L,  * 


b  -  Pe 


1  [Fe(l  -  Pe)M]1/2 

The  error  in  Eq.  (11)  is  such  that  the  normal  approximation  is  small  by  a  negligible 
amount  when  n  Pe  (1  -  Pe)  >  25.  Thus  from  Eq.  (9)  Pe2  must  be  such  that 


£  *  1 
2  *  VT5T 


U  2 

/  2  e'1'2'  dt 


(12) 


where 


,,  P'e  -  Pe2  , 

U2=  [Pe2(l  -  Pe2)/n} 1/2 


,  -  Pe  a _ 

2  fPe2  (1  -  Pe2Vn  ]  V2 


Under  the  conditions  n  Pe2(l  -  Pe)  >  25  ami  Pe0<  0. 1 


L2  -  -  Vn  Pe2  <  -5 


Thus  with  these  restrictions  on  n  and  Pe^,  Eq.  (12)  can  be  approximated  by 

£  1 

2  *  /sir 


1  Ju2  e-l/2 12 


dt 


(13) 


8. 


Introducing  similar  approximations  into  Eq.  (10)  shows  that  Pe^  must  be  such  that 


£  1  r“  - 1/2 12 


2  sj  2k’ 


/  e  dt 

l3 


(14) 


where 


Lo  -  - 


J3  “  [ Pex  (l  -  Pep/n]  V* 


With  K€  =  defined  by 


£ 
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_L_  /' 

nT2F  jk 


•  1/2 12 


dt 


(15) 


These  relations  for  Pe^  and  Pe^  become 


P*e  -  Pe2 

(Pb2  (l-Pe2)/njV2 


=  -  K 


fre  -  Pel 

[PBjd-Pep/n]1^ 


Using  again  the  conditions  n  Pe  (1-Pe)  >  25  and  Pe  <  0. 1  yields  the  final  results 


Pei  ®  Pe'  ( 1  -  K€  x  l/^JnW'] 


Pe2  -  Pfe’  |l+K£xl />TnW'] 


(16) 


9. 


These  results  are  plotted  in  Figures  1  and  2  for  c  *  0. 05  and  0. 01  respectively 
and  examples  of  their  use  are  given  in  Section  II-5. 


2.  Confidence  Intervals  for  Noise  Power 

The  estimate  of  the  noise  power  NQ  is  given  by 


fl  * 
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-2 
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1 

n  -  1 
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i*l 


/  -  v2 

(x1  -  M  ) 


(17) 


where 


or 

It  is  known  [2]  that  the  quantity  —7-  (n  -  1)  is  distributed  by  the  chi-square  distribu¬ 
te 

tion  with  (n  -  1)  degrees  of  freedom  i.e. , 


where 


p(x2) 
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ri  -  1 
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) 


2  ,  , 

\  =  (n-D 


(2)'  (fl*1)/2  (X2)(”'3)/2  e'V*2 


(n  -  1) 


2 

xr  > 


(18) 


Now,  from  Eq.  (5)  the  upper  confidence  limit  is  the  value  of  Nq  such  that 


p(vJ)  d  x2 


e 

2 


(19) 


In  [6]  it  is  shown  that  the  distribution  for  x  is  approximately  Gaussian  with  mean 
n  -  1  and  variance  for  n  as  small  as  5.  Thus  Eq.  (19)  can  be  approximated  by 
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-  «0 


e 

2‘ 


(20) 


10. 


where 


u  =  N/?r~ T  [^P'Nq-1] 

4  i/n/T 


With  Kg  as  in  Eq.  (15)  the  final  expression  for  is  thus 


N  =  ST  (  1  -  -Ki  _  ]"2 
02  0  ^1^1) 


(21) 


Similarly  the  lower  confidence  limit  is  found  to  be 


N  ,  =  N’  (  1  + 
ol  o 


-2 


V2(n-1) 


(22) 


For  n  >  30,  the  error  in  these  confidence  limits  due  to  the  approximation  of  Eq.  (20) 
is  less  than  1%.  Figure  3  presents  these  results  graphically. 

3.  Confidence  Intervals  for  Signal  Power 


The  maximum -likelihood  estimate  of  the  signal  power  Sq  is  [1] 


som;>2  = 


1  Z  *■ 

n  i=i  1 


From  Eq.  (6a)  the  lower  confidence  limit  S^  must  be  such  that 


P  {  §  <  S’  f  S  . }  *  1  -  e/2 
r  1  o  o  '  ol 1 


(23) 


Thus  since  p  can  be  negative,  Eq.  (23)  becomes 


P  {  -  n/T"  <  p  <  V S7 1  S  ,  }  *  1  -  e/2 

r  o  r  o  ol  1 


(24) 


11. 


It  is  known  [2]  that 


t 

c 

is  distributed  by  the  t -distribution  with  n  -  1  degrees  of  freedom, 
square  of  the  value  of  p  such  that 
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ff/vn 


Thus  S .  is  the 
ol 


where 


1-r 
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p  (t,  n  -  1)  dt 


(25) 
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4F- i 


*/sT 


*  a  /'s/Jr 


p(t,  n  -  1)  »  the  central  ^distribution  with  n  -  1  degrees  of  freedom 


To  simplify  this  implicit  relation  for  \i  rewrite  Eq.  (25)  as 


u5 

/  p(t,  n  -  1)  dt 


+ 


fL  5 

J  P  (t,  n 


1)  dt  »  2  -  t/2 


(26) 


Since  this  expression  is  even  in  p,  p  >  0  are  the  only  values  of  interest. 

Requiring  /o'  >5  and  n  >  10  makes  the  second  integral  in  Eq.  (26) 

equal  unity  to  four  decimals.  Under  these  conditions  Eq.  (26)  can  be  replaced  by 


-  00 


n  -  1)  dt 


1  -  e/2 


(27) 


12- 


Defining  t  (n)  by 

,Mn> 

J  p(t,  n-l)dt=  l-c/2 


(28) 


gives  for  the  lower  confidence  limit 


A  similar  procedure  gives 


where 

p’«  tfyfr’]2  -  s;/n; 

For  n  >  120  the  t-distribution  is  closely  approximated  by  the  normal  distribution  with 
mean  zero  and  variance  unity.  Thus  when  n  >  120,  t,(n)  *  K.  may  be  used  in  Eqs.  (29) 

c  c 

and  (30).  Figures  4  and  5  illustrate  these  Tesults. 

4.  Confidence  Intervals  for  Signal -to -Noise  Ratio 
The  estimate  of  the  SNR,  p,  is  defined  by 
P  =  I  u  /a]2  «  So/fl0 

Thus  from  Bq.  (6)  the  lower  confidence  limit  p^  must  satisfy 

Pr  {  p  <  p’  I  p  j  }  »  1  -  e/2  (31) 


or,  since  p  can  be  negative, 


Pr  {  -  >Jpr <  £  <  •7pr  |  Pj  }  *  1  -  c/2 


(32) 
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Now,  the  quantity 


where 
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2+6 

sfvT 


7- 


a  ul 

a/sftT 


6  «7pT 

or 


w  * 


is  known  [7]  [8]  to  be  distributed  by  the  non-central  t -distribution  with  n  -  1  degrees 
of  freedom.  Thus  is  found  by  determining  a  value  of  6  such  that 


p(t,  6 ,  n  -  l)dt  * 


1  -  e  /2 


(33) 


where  p  (t,  6  ,  n  -  1)  is  the  non-central  t -distribution  with  n  -  1  degrees  of  freedom. 
Unfortunately,  tables  of  the  non -central  t -distribution  are  not  suited  for  calculations 
of  this  type  since  trial  and  error  determination  of  6  is  involved.  This  difficulty  can 
be  circumvented  in  the  following  manner. 

With  the  above  definitions  of  t ,  6  ,  and  w,  and  defining  6^  to  be  the  value  of  6 
satisfying  Bq.  (33),  Bq.  @2)  can  be  rewritten  as 


p  {  -*/n~p’  <  — 6  i  <  Sp*  }  ■  1  -  e/2 

7w" 


(34) 


or 

Pl+  p2  *  1  -  e/2 


«5) 
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where 


Fj  =  Pr  {  -z  -  s/np'  <  -  z  +  N/np*  Vw~  (  *7w~ >  0  }  Pr{  n/w~ >  0  } 

^2  =  Pr  ^  ~z  +  ^nP'  <  6  j  <  -  z  -  N/np’  n/w"  f  n/w- <  0  }  Pr  {  Vw” >  0  } 

(At  this  point  it  is  convenient  to  ignore  the  fact  that  Pr  {*Vw" <  0}  =  0,  so  that  the 
following  discussion  may  be  simplified.  This  fact  will  be  reintroduced  at  a  later  point. ) 

Now,  using  the  joint  density  function  for  z  and  n/w  ,  P^  and  P2  can  be  found  by 
integrating  this  over  the  regions  of  the  z ,  ‘JvT  plane  indicated  below. 


z 


However,  the  difficulty  in  evaluating  this  double  integral  for  general  n  makes  it 
convenient  to  determine  these  probabilities  in  the  following  manner.  Let 

x  e  -  z  -  Vnpr  •JvT 

y  S  -  z  +  Vnp’  Vw 

It  is  known  [2]  that  z  is  normally  distributed  with  mean  zero  and  variance  unity,  and 
that  (from  Sect.  II- 2)  n/w  is  approximately  normally  distributed  with  mean  unity  and 
variance  [2(n-l>]  It  is  also  known  that  the  normal  approximation 
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for  the  distribution  of  nTw"  is  quite  accurate  even  for  small  n  and  that  z  and  Ww~  are 

statistically  independent  [2] .  Therefore  the  distribution  for  -x  and  y  is  even  more 

accurately  approximated  by  the  normal  distribution  with  mean^n  p'  and  variance 

1  +np'  [2(n  -  1)]  1.  Thus,  this  approximation  allows  probabilities  such  as 

Pr  {  x<X}  or  Pr  {  y  <  Y  }  to  be  accurately  determined  using  tables  of  the  normal 

distribution.  It  will  now  be  shown  that  P  {  x  <  6  }  -  P  {y<6.}ssP.  -P_,  when 

it  i  r  l  i  z 

Pj  and  P^  are  as  previously  defined,  and  thus  that  the  normal  distribution  may  be  used 
to  evaluate  Bq.  (33). 


Thinking  again  in  terms  of  the  joint  density  function  for  t  andsTvr ,  P^{y  <  Y} 
would  be  obtained  by  integrating  this  over  the  region  in  the  t ,  *FtP  plane  below. 
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known  that  Pr  {n/w" <  0}  =  0.  Therefore  if  the  normal  approximation  to  the  distri¬ 
bution  of  *'/w”  is  to  be  accurate  P  { "'/w* <  0}  calculated  from  it  must  be  negligibly  small. 
Requiring  (2(n-l)]  _1/2  <  1/4,  or  equivalently  n  >  9,  makes  Pr  { 'VvT <  0}  and  thus 
?2  less  than  10’4.  With  this  restriction  on  n,  Eq.  (35)  becomes,  with  negligible 
error, 

Pr  {x  <  fij}  -  Pr  {y  <  6X}  =  1  -  e/2  (36) 

Introducing  the  normal  distribution  and  recalling  that  62  =  npj,  Eq.  (36)  is  evaluated 
as 

—  e  1/7x2  dt  -  -p=r  /  6  e‘1/2t2dt  =  1  -  e/2  (37) 

sIW  -»  ^27  -00 


where 


‘Jnpl  +  slop' 

U6  =  {  1+np1  (2(n-l)]'1}l/2 


u  =  ^ 

7  { 1+np  [ 2(n- 1) ]  ' *  }  i/2 


Since  *v/n pj  a  0  the  requirement 


_ *7np' 

(l+p-  [2(n-o  rH1/2 


>  4 


or  approximately 

n>8  (1+  -|r  ] 

makes  the  first  integral  in  Bq.  (37)  equal  unity  to  4  decimals.  Therefore,  under  the 
double  restriction 

n  >  9 

n>8  (1+  J7] 
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the  lower  confidence  limit  p.^  is  found  from 


U-, 


/  7  e'1/2t  d. 
VTF  .  so 


£ 

2 


(38) 


In  terms  of  as  previously  defined  the  solution  is  thus 


A  similar  procedure  shows  that 


Figures  (6)  through  (9)  present  these  results  graphically. 

5.  Examples 

a)  The  output  of  a  digital  receiver  is  observed  for  6. 0  x  10*  bauds » .and  24 
errors  are  observed. 

n  ■  6. 0  x  10* 
ffe«  4  x  10'* 

~4  *4 

From  Figure  1,  2.4  x  10  <  Pe  <  5.6  x  10  with  a  confidence  level  of  95%. 

b)  It  is  desired  to  measure  a  probability  of  error  which  is  expected  to  be  about 
10  with  a  confidence  level  of  95%  and  a  confidence  interval  of  +  10%  of  the  measured 
value.  How  many  samples  should  be  used?  From  Fig.  1  a  measured  value  of 
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-3  5 

Pe  =  10  with  n  *  4  x  10  samples  would  have  the  desired  confidence  level  and 
interval.  Therefore  an  approximate  sample  size  of  4  x  10  should  be  used.  Note, 
however ,  that  the  actual  confidence  interval  must  be  determined  after  Pe  has  been 
measured. 


c)  A  sample  of  n  -  1000  gives 

§’  =1.0 

o 

a*  =4.0 

o 

p'  =  0.25 

From  Figs.  4  ,  3,  and  6  respectively  the  95%  confidence  intervals  are 

0.77  <  S  <  1.26 

-  o  — 

3.68  <  N  <  4.4 

-  o  — 

0.21  <  p  <  0.33 


d)  It  is  desired  to  measure  an  unknown  noise  power  to  within  approx¬ 
imately  +  10%  at  a  99%  confidence  level.  From  Fig.  5  a  sample  size  of  1. 7  x  10^  gives 

a  confidence  interval  of  + 10%,  -  9%  for  any  value  of  1^. 


e)  As  a  less  trivial  example  of  the  use  of  confidence  intervals,  consider 
the  problem  of  evaluating  a  low  data  rate  binary  communication  system  having  the 
following  theoretical  performance  characteristics. 


-I.* 


SNR  =  p  (  1  +  1/2/3  ] 


-1 


/3  a  0 
^0 


where  /3  =  —  =  Slgnal  enerBY  baud  at  receiver  input 
Noise  spectral  density  at  receiver  input 


(41) 

(42) 
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The  system  is  to  be  evaluated  by  measuring  either  P  or  SNR  and  calcu- 
lating  an  "effective"  0  from  Eq.  (41)  or  (42)  respectively.  Comparison  of  the  calculated 
0  with  the  value  actually  used  in  the  test  gives  an  indication  of  the  departure  of  the  system 
from  ideal  operation.  The  question  arises  as  to  which  of  the  two  measurement  methods 
provides  the  desired  measurement  accuracy  with  the  least  number  of  samples.  The 
following  analysis  demonstrates  that  for  small  0  the  two  methods  are  approximately 
equivalent  while  for  large  0  the  number  of  samples  required  for  SNR  measurements  is 
less,  by  several  orders  of  magnitude,  than  the  number  required  for  P  measurements. 

Assume  that  the  probability  of  an  error  is  independent  from  baud  to  baud, 
that  the  noise  component  of  the  samples  used  in  calculating  SNR  is  independent  from 
baud  to  baud,  and  normally  distributed,  and  that  the  desired  accuracy  for  the  calculated 
"effective"  0  (£)  is  +  100a%  at  a  confidence  level  of  100(1- 1)%.  Then,  Eq.  (16)  shows 
that  the  number  of  samples  used  for  measurements  must  be  at  least  as  large  as  the 
largest  value  of  n  satisfying  the  relation 

fll  +  K/VnTM  =  Ie'^ll  +  al  (43) 

e  —  €  e  2 

where  0’  =  g  e  ^ 
e  2 

For  a  given  a  this  equation  can  be  solved  for  n.  Choosing  the  sign 
yielding  the  largest  n  gives  the  following  conservative  result. 

n  2  2K?  e^  l  1  -  e^“  ]  2  (44) 

This  result  is  {dotted  in  Fig.  10  for  values  of  a  corresponding  to  errors 
in  0  of  +  0. 2  db  and  +0.4  db. 
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From  Eqs.  (39),  (40)  and  (42)  the  number  of  samples  used  for  SNR  (p)  measure¬ 


ments  must  be  at  least  as  large  as  the  largest  value  of  n  satisfying 

1 12  . 


!  L  +  jl_ 

■Jf,'  2(n-l) 


=  /3(l+a) 


1  + 


1 


20(l+a) 


(45) 


where  p'  =  B  (  1+  —  ] 
2/3 

Now ,  for  0  >  2 


I  1  + 


2/3  (1+a) 

Similarly  for  n  >  1000 


f1  ~  1 


9 

. 

m 

i+5t 

-  vn  * 

/  i  +  n 

£ 

1 

1  “  \Tn  ' 

1 1  +  r 

/  p'  2(n-l) 

— 

J  p’  +  2 

. 

- 

J 

-1  2 


Thus  for  0  >2  and  n  >  1000  Eq.  (45)  becomes  approximately 


P' 


1  + 


JF? 


*  £<l+a) 


—  *7n  -v  p- 

Solving  this  for  the  largest  value  of  n  gives 


n21y  1 

[1  -  n/i-c*  ^'J 


(46) 


Fig.  10  illustrates  this  result  for  errors  in  /§  of  +  0.  2  db  and  +  0. 4  db. 

Heuristic  reasoning  suggests  that  due  to  the  3mall  noise  variance  at  high  SNR 
(and  thus  low  Pg)  it  should  be  possible  to  measure  $  via  SNR  with  relatively  few  samples 
as  compared  to  measurement  via  Pg.  Fig.  10  provides  a  striking  confirmation  of  this 
conjecture  and  demonstrates  that  for  /3  >  2  a  given  accuracy  can  always  be  obtained  with 
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fewer  samples  via  SNR  measurements.  Even  more  important,  however,  is  the  fact 
that  for  /?  on  the  order  of  10  or  greater  (corresponding  to  P  <  10'^)  the  reduction  in 
sample  size  is  several  orders  of  magnitude.  This  fact  coupled  with  accurate  knowledge 
of  the  measurement  accuracy  for  a  given  sample  size  allows  meaningful  evaluation  of 
system  performance  with  an  absolute  minimum  of  samples . 

III!  CONTINUOUS  ESTIMATION  ERRORS 


This  section  is  concerned  with  the  errors  involved  in  the  estimation  of  the  mean 
square  value  of  an  ergodic  Gaussian  random  process.  The  continuous  and  time -varying 
estimate  considered  here  is  obtained  from  a  finite  time  average  of  the  square  of  a  sample 
function  of  the  process.  A  true  RMS  voltmeter  is  an  example  of  a  device  utilizing  this 
method  of  measurement;  the  meter  indicates  the  instantaneous  output  of  a  low  pass  filter 
that  provides  the  weighted  time  average.  Due  to  the  finite  averaging  time  (or  equivalently 
the  non-zero  filter  bandwidth)  the  estimate  varies  with  time;  the  rate  of  variation  being 
inversely  proportional  to  the  bandwidth  of  the  process.  The  following  analysis  will 
obtain  probabilistic  bounds  on  the  magnitude  of  these  variations  that  are  roughly  analogous 
to  the  confidence  interval  relations  for  discrete  estimates  discussed  in  the  previous 
section. 

Consider  the  following  circuit  which  provides  the  estimate  described  above. 


s(t) 


estimate  of  s 


(t> 


From  the  Superposition  Integral  it  follows  that 

00 

en(t)  *  /  s2(t)  h(t-r)dr 

U  -00 

Thus  the  average  value  of  eQ(t)  (either  ensemble  or  infinite  time)  is 
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E  [e0(t)]  2  M 


=  E  [  f  s2(r)  h(t-r)  cJt  ] 


=  E[  s2(t)  ]  H(0)  =  R(0)H(0) 


(47) 


Assuming  that  H(t)  is  normalized  (i.e.  H(O)  =  1)  this  demonstrates  that  the  average 
value  of  the  estimate  is  equal  to  the  mean  square  value  of  the  input  signal.  However, 
due  to  the  finite  filter  bandwidth,  eQ(t)  will  vary  about  its  mean  value.  These  variations 
are  most  conveniently  investigated  by  determining  the  variance  of  eQ(t). 

By  definition 


where 


Var(e0(t)]  =  E(e02(t)]  -  E[eQ(t))2  =  e2(t)  -  R2(0) 

Since  s(t)  is  assumed  to  be  Gaussian  it  is  known  that  [  9] 

_  00 

e2(t>  =  /  S.(f)  | H(f)  1 2  df 


S.(f)  *  R2(0)  6  (f)  +  2  /  S(f  )  S(f-f* )  df‘ 

.00 

6  (f)  =  unit  impulse  function 

S(f)  =  power  spectral  density  of  s(t) 


Thus 


Var[eo(t)]  h  a2  =  2  /  /  S(f ’)  S(f-f )  |  H(f)|2  df’  df 

-00  -oo 


or,  equivalently 


o2  *  2  /  /  1  H(f)  J2  e‘i2irtT  R2  (t)  drdt 


(48) 


(49) 


where 


R(t)  =  R(-t)  *  /  S  (f)  ei2TfT  dr 


(50) 
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Thinking  in  terms  of  the  RMS  voltmeter,  a  convenient  method  for  using  this 
variance  to  indicate  the  estimation  error  is  to  specify  in  db  (relative  to  the  average 
reading)  the  value  corresponding  to  three  standard  deviations  above  the  average  reading. 
Let  this  estimation  error  or  deviation  parameter  be  denoted  by  D.  Then  from  above 

D  =  101og1()  (  1+  ~]  db  (51) 

For  D  to  be  small  it  is  necessary  that  the  filter  bandwidth  be  narrow  with 
respect  to  the  signal  bandwidth.  However,  under  this  condition  the  filter  output  is 
approximately  Gaussian.  Thus  for  small  D  the  meter  reading  will  be  within  +  D  db 
of  the  correct  value  more  than  99%  of  the  time.  Considered  in  this  manner,  D  is 
roughly  equivalent  to  the  confidence  interval  for  noise  power  discussed  in  the  preceding 
section. 

Due  to  the  difficulty  in  evaluating  Eq.  (48)  for  arbitrary  S(f)  and  H(f)  it  is 
desirable  to  provide  an  upper  bound  to  Eq.  (51)  that  does  not  involve  the  double  integral 
of  Eq.  (48).  This  is  readily  accomplished  as  follows.  The  inner  integration  of  Eq.  (48) 
is  the  convolution  of  S(f)  with  itself.  Let  the  resulting  spectrum  be  Sj(f).  Then  from 
the  Schwartz  inequality  it  follows  that  * 


Sj(f)  <  S^O) 

2 

Thus  o  can  be  upper  bounded  by 

_2  nl.  .nv  C  f  I  .... 


f  *0 


<*  <  2Sj(0)  /  lH(f)fdf  =  2  /  S2(f)df  /  |H(f')f  df’ 


(52) 


*  By  definition  Sj(f)  =  /  S(f')  S(f-f')  df', 


and 


Sl(0)  =  /  s2(f)df  =  £  s2(f-f*)df* 


Thus  from  the  Schwartz  inequality 

•O 


[/  S(f')  S(f-f')  df’  ]2  St(f) 

/  S2(f’)  df’  /  S  tf-f')df’  Sf  (0) 

where  the  equality  holds  if  and  only  if  S(f’)  =  S(f-f),  i.e.  when  f=0.  It  should  be  noted, 
however,  that  physical  reasoning  shows  that  Eq.  (52)  will  introduce  negligible  error  when 
the  bandwidth  of  S(f)  is  much  greater  than  that  of  the  filter. 
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Since  H(f)  is  assumed  to  be  normalized  the  second  integral  in  Eq.  (52)  defines  the  noise 
bandwidth  W  of  the  filter.  In  this  discussion  it  is  convenient  to  define  the  "effective 

n  i 

“1  o 

integration  time  constant”  of  the  filter  by  T»(2  Wn)  .  Then  an  upper  bound  for  (a/ nr 
can  be  written  as 


a2 


f  ”  S2(f)  df 
ljL  S(f)  df  ] 


7 


Define  a  signal  "  noise  bandwidth"  by 


S(f) 

max  S(f) 


df 


(53) 


(54) 


Then  Eq.  (53)  can  be  written  as 


a2  T 
M7  "  W? 


rT-ggL.  1 

J.„  [.max  S(f)J 


df 


Now,  S(f)  is  by  definition  a  non-negative  function.  Thus  it  follows  that 


[.Ml  1 

LmaxS(f)J 


S(f) 


_max  S(f)J  max  S(f) 
and  therefore  that 

"S(f) 


s  l 


f  "f  gffl— 1 

.«LmaxS^  J 


max  S(f)  |  df  S  WS 


Substituting  this  result  into  Eq.  (53)  yields 


(55) 


TW„ 


(56) 


and  thus  from  Eq.  (51) 


D  <  10  log  1Q  [  l+3(^-)1/2J  db 

s 


with  H(0)  =  1,  where  Wg  is  defined  by  Eq,  (54)  and, 
T  =  (  /  |H(f)  |2  dff 1 


(57) 


(58) 


25. 


Eq.  (57)  is  plotted  as  curve  1  in  Figure  11  along  an  exact  expression  derived  in  the 
example  below.  These  curves  indicate  that  Bq.  (57)  provides  a  convenient  and  sufficiently 
accurate  method  for  determining  the  required  filter  ’’integration  time  constant"  Twhere 
a  spectrum  shape  and  a  measurement  accuracy  are  given. 


Example  1 
Let 

H(f)  =  [  l+j2irT,  f  ]  1 

S(f)  =1  (f|  <  W 

0  |f|  >  W 

Consider  first  the  exact  calculations  via  Eqs.  (47),  (48)  and  (51). 
00 

p  =  J  S(f)  df  =  2  W 
.00 

<r2=  2/*°/  S(f’)S(f-f’)  |H(f){2  df  df' 


r2W  f  2W-f] 

=  4  {  1+4*^  df 


-  ^  Tan'1[4trT1W] . In  [  1+ 16tt2t2  W2] 


D  =  101og1()(l  +  3-p] 


*  101<*io  {1  +  3  T*n‘‘f’Tiw-  1  <St^T>2i"11+1^w2i  j} 


(59) 


For  TW  >  10  Eq.  (59)  becomes,  with  negligible  error, 


D  =  ioiog10ll  +  3(^-)1/2]  db 


(60) 
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The  upper  bound  of  Eq.  (57)  is  evaluated  for  this  example  as 
Ds  101og10  [l+3(^r-)1/2)  db 


where 


2  -1 


T  ®  [2  J  |H<f>r  df  ] 


=  12  / 


l+4ff2T?fi 


W  =  /  S(f)  df  =  2W 
^  .00 

Thus  the  upper  baud  of  Eq.  (61)  and  the  asymptotic  expression  of  Eq.  (60)  are 
identical  when  S(f)  is  a  rectangular  spectrum.  This  is  in  complete  agreement  with  the 
approximations  used  in  deriving  Eq.  (58). 

Example  II 


S(f)  =  l  l  +  (jrf/2W)2] 


H(f)  *  ( 1+j  2wTjf  ] 


The  exact  analysis  gives 

oo 

M  =  /  S(f)  df  =  2W 


R(t)  *  /  S(f)  e^2irfT  df  =  2W  e 


"4W|  T  | 


Prom  Eq.  (49)  the  variance  is 


o1  =  2  Rh(t)  R2(t)  dr 


where 


00 


RH<T>  =  tH<f>t2e‘j7rfTdf 

For  this  H(f) 

R  (y )  a  _ L.  e '^~T  1 
VT)  2T 

Thus 

9  W 2  r°°  1 

a2  =  8  -j:  J  exp  -  [8W  +  -  ]  t  dr 

>00 

_  8W2 
=  8WT+1 


and 

D  =  10  los,0  1  iwTTi  >  1/21  db 


(62) 


For  TW  >  10  Eq.  (62)  becomes,  with  negligible  error 


D  =  10  lcg1() 


[  1+3 


]  db 


(63) 


Evaluating  T  and  W_  for  the  upper  bound  gives  T  =  T  and  W  =  2W.  Thus  the 
^  *  5  ^ 

asymptotic  expression,  Eq.  (63),  differs  from  the  upper  bound  by  a  factor  of  ~  under 
the  radical.  For  large  TW^  this  results  in  a  negligible  difference  between  the 
respective  values  of  D.  Eq.  (62)  is  plotted  vs  TWg  as  curve  2  in  Fig.  11. 


IV.  SUMMARY 

In  this  report  the  statistical  concept  of  confidence  intervals  has  been  applied  to  the 
problem  of  obtaining  probabilistic  bounds  on  the  errors  arising  when  a  finite  number  of 
samples  are  used  to  estimate  the  following  parameters  of  a  Gaussian  random  process. 

a)  the  squared  mean- -called  signal  power, 

b)  the  variance --called  noise  power, 

c)  the  ratio  of  squared  mean  to  variance— called  SNR, 

d)  the  probability  that  the  sample  value  exceeds  an  arbitrary  threshold --called  P  . 
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These  results  are  useful  in  evaluating  digital  communication  systems  by  providing 
information  as  the  accuracy  of  test  measurements  and  by  allowing  tests  to  be  designed 
to  provide  the  required  accuracy  with  a  minimum  of  data.  An  example  has  been  given 
in  which  the  use  of  these  results  allows  a  reduction,  by  several  orders  of  magnitude, 
in  the  amount  of  data  required  to  obtain  a  specified  accuracy. 

A  bound  roughly  analogous  to  the  confidence  interval  for  the  discrete  estimate  has 
been  obtained  for  continuous  estimates  (e.g.  an  RMS  voltmeter)  of  noise  power.  The 
bound  is  a  function  only  of  the  noise  bandwidth  of  the  averaging  filter  and  of  the  signal 
and  is  shown  to  be  quite  accurate  when  the  measurement  accuracy  is  reasonably 
small  (<  0. 5  db). 
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